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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-8 and 19 is/are rejected. 

7) [X] Claim(s) 9-18 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)[X] The drawing(s) filed on 05 February 2002 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)KI All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3-D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachments) 

1 ) [3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) |3 Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) d Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date 0202 . 6) D Other: . 
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DETAILED ACTION 
Drawings 

1. The informal drawings are not of sufficient quality to permit examination of the 
drawings. Accordingly, replacement drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to this Office action. 

Claim Objections 

2. Claim 14 is objected to because of the following informality: 

The phrase "and preferably ..." should be removed or positively recited because it is not 
clear whether the limitation is required. Appropriate correction is required. 

3. Claims 1 1, 12, 16, and 17 are objected to because of the following informality: 
In claim 1 1, the variable Nj should be defined. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

5. Claims 1-6 are rejected under 35 U.S.C. 102(b) as being anticipated by "Hybrid Image 
Segmentation Using Watersheds and Fast Region Merging" by Haris et al. ("Haris")- 

Regarding claim 1, Haris discloses a method for the fragmentation of images into 
homogeneous regions, this fragmentation using iterative merges of fragments Fj and Fj (i.e. two 
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regions R*m and R* j M ) which are as similar as possible according to at least one selection 
parameter, this similarity being evaluated by a product A*B of two factors A and B (see equation 
(12), page 1689), A being consistent with a number of pixels (A = ||R*m|| * HR* j M|| ) and B being 
consistent with the selection parameters (B = QifllR'Mll) - H (l|R* j M||)] 2 ), characterized in that a 
merge is performed when the product A*B*C is less than a threshold consistent with the 
selection parameter, C being a factor consistent with the inverse of a number of pixels (C = 1 / 
(l|R\.|| + l|R* j Mll) )• 

Regarding claim 2, Haris discloses that C is proportional to 2/N m where N m represents a 
mean size of the fragments Fi, such as a mean number of pixels (i.e. C represent the inverse of 
the sum of the cardinality of sets R*' M and R* j M and is proportional to the inverse of the mean 
number of pixels). 

Regarding claim 3, Haris discloses the selection parameter(s) is (are) at least luminance 
(i.e. the luminance Y is used in the calculation of B). 

Regarding claim 4, Haris discloses that each iterative merge relates only to two 
neighbouring fragments Fi and Fj ( 7(i,j) in equation (12) shows that the merge only relates to 
adjacent regions). 

Regarding claim 5, Haris discloses that the factor A is proportional to the claimed 
equation (i.e. A = ||R*m|| * I|R* j m||, which is proportional to the given equation). 
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Regarding claim 6, Haris discloses that the size of a fragment is the number of pixels 
included in this fragment (i.e. the size of each region is the cardinality of the region - that is, the 
number of pixels in the region). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Harris in view of . 

Regarding claim 7, Haris discloses that the factor B is proportional to (Yi - Yj) 2 , where 
(Yj - Yj) represent the difference between the luminances of each of these two fragments (see 
equation (12), page 1689). 

Haris does not disclose that the U and V components are utilized because Haris 5 
disclosure is in terms of a greyscale image. 

Moed discloses an image segmentation system that operates on a color image. In 
particular, Moed teaching utilizing the YUV color format to segment an image, and the 
segmentation algorithm involves generating Y, U, and V difference information. See column 4, 
lines 18-48. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Haris by Moed to segment a color image in the YUV format and utilize the squared 
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difference of the U and V channels, since Moed discloses that it is conventional to perform color 
segmentation on YUV images. 

8. Claims 8 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Matching Error Based Criterion of Region Merging for Joint Motion Estimation and 
Segmentation Techniques" by Schiitz et al. ("Schiitz") in view of Haris. 

Regarding claim 8, Schiitz discloses a method of grouping fragments of an image (i.e. 
regions), characterized in that this grouping uses a model of motion individual to each fragment 
Fj (§ 3 - the motion M of each region is estimated and used to group, or merge, the regions 
together), this model of motion is determined with a known error or variance so as to allocate to 
the fragment created by this grouping one and the same model of motion (§ 4 - the motion of 
model for each grouped (merged) region is determined with the known mean absolute error 
(MAE) of the regions previously computed), grouping together at each step of this method the 
fragments Fj and Fj which minimize a grouping cost proportional to an evaluation of the 
difference between the models of motion of the two fragments Fj and Fj (§ 3 - the difference in 
the motion models, M A and M B , of two regions, A and B, is determined on the basis of the mean 
absolute error (MAE); § 4 - the MAEs between each pair of regions are evaluated to determine 
which adjacent regions exhibit the lowest merging cost, and then those regions whose grouping 
costs were determined to be smallest are merged). 

Schutz discloses that the initial regions are obtained by a spatial watershed segmentation 
(see § 2). 
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However, Schiitz does not disclose that the initial fragments (regions) are obtained 
according to the method of claim 1. 

Haris discloses the method of spatial segmentation according to claim 1. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to modify Schiitz by 
Haris to create the initial segmentation according to claim 1, since Schiitz discloses that the 
initial segmentation of the regions is accomplished through a spatial watershed segmentation 
algorithm, and Haris shows that a watershed segmentation that employs the details of claim 1 is a 
conventional way of spatially segmenting an image into homogeneous regions. 

Regarding claim 19, Schiitz discloses the stopping condition is given by a minimal 
threshold (see § 4). 

Allowable Subject Matter 
9. Claims 9-18 are objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

Regarding claim 9, Schiitz does not disclose: (1) that the grouping cost is inversely 
proportional to a threshold of motion, or (2) that in the absence of any other stoppage test, the 
grouping is not allowed if the evaluation is greater than this threshold. Rather, Schiitz teaches 
determining the grouping costs between each pair of adjacent regions and merging regions that 
have a minimal cost (see § 4). 
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Regarding claim 11, Schutz does not disclose the claimed details of the delta equations 
that are used to calculate the evaluation of the difference of motion between two fragments. 

Regarding claim 18, Schutz is silent to the claimed stoppage test formula. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

U.S. Patent 5,557,684 by Wang et al. 

"Motion-Based Segmentation Using a Thresholded Merging Strategy on Watershed 
Segments" by Smet et al. ^ 

"Motion Estimation and Representation for Arbitrarily Shaped Image Regions" by 
Karczewicz et al. 

"Spatiotemporal Segmentation Based on Region Merging" by Mosceni et al. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Amelia 
Au, can be reached on (703) 308-6604. The fax phone number for the organization where this 
application or proceeding is assigned is (703) 872-9306. 
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Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 
306-0377. 
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Group Art Unit 2623 
20 November 2004 




